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Discussions  of  acquisition  reform  over  the  past  twenty- five  years  have  usually  put  the 
Department  of  Defense  (DoD)  Program  Manager  (PM)  and  personnel  in  the  program  office  in 
the  foreground.  These  people  oversee  the  contractors  and  do  myriad  things  that  must  be  done  by 
the  government  for  a  major  acquisition  program  to  move  forward — contracting,  financial 
management,  and  test  planning,  among  many  others.  In  the  background  are  the  contractors  who 
typically  do  the  development  and  manufacturing.  A  good  program  will  not  occur  if  the 
government  personnel  and  contractors  do  not  do  their  jobs  well.  It  is  equally  true  that  if  these 
individuals  and  organizations  do  their  jobs  well,  a  good  outcome  for  the  program  is  more  likely. 

What  this  focus  on  the  DoD  PM,  the  program  office  personnel,  and  the  contractors’  PMs 
and  workers  leaves  out  are  factors  they  must  accept  as  “givens.”  These  givens  are  subject  to 
changes — sometimes  large  and  fairly  sudden — that  presumably  have  substantial  consequences 
for  program  outcomes.  One  of  the  givens  is  DoD  acquisition  policy  and  process.  A  second  is  the 
DoD  budget,  which  does  not  determine,  but  generally  has  a  marked  influence  on,  the  finding  for 
individual  programs. 

This  paper  summarizes  the  results  of  research  done  at  the  Institute  for  Defense  Analyses1  on 
whether,  taking  account  of  finding  climate,  changes  in  DoD  acquisition  policy  and  process  had  a 
discernible  effect  on  growth  of  Program  Acquisition  Unit  Cost  (PAUC)  of  major  defense 
acquisition  programs  (MDAPs).  The  paper  attempts  to  shed  some  light  on — and  as  it  turns  out,  to 
challenge — three  assumptions  that  often  are  implicit  in  discussions  of  acquisition  reform 

•  Cost  growth  and  other  problems  of  program  outcome  primarily  reflect  what  does  and 
perhaps  does  not  take  place  in  the  acquisition  realm 

•  Changes  in  acquisition  policy  and  process  can  be  expected  to  have  substantial  effects  on 
the  cost,  schedule,  and  performance  results  of  MDAPs. 

•  The  ingrained  cultures  of  DoD  acquisition  organizations  are  a  substantial  obstacle  to 
steps  that  might  reduce  cost  growth. 

While  not  organized  around  these  topics,  the  paper  provides  results  that  bear  directly  on  them 

Acquisition  Regime  and  PAUC  Growth 

DoD  acquisition  policy  and  process  over  the  period  1970-2007  can  be  grouped  into  five 
successive  regimes: 

1.  The  Defense  Systems  Acquisition  Review  Council  (DSARC),  1970-1982 

2.  The  Post-Carlucci  Initiatives  DSARC,  1983-1989 

3.  The  Defense  Acquisition  Board  (DAB),  1990-1993 

4.  Acquisition  Reform  (AR),  1994-2000 

5.  The  DAB  -  Post  Acquisition  Reform  2001-2007 


*  Research  Staff  Member,  Institute  for  Defense  Analyses.  Valuable  comments  reflected  in  the  paper  were  provided 
by  David  Sparrow,  Brian  Rieksts,  and  Prashant  Patel.  Sarah  Bums  provided  advice  and  assistance  on  the 
statistical  analysis.  Linda  Wumanaged  data  acquisition  and  the  database. 

1  David  L.  McNicol  and  Linda  Wu,  “Evidence  on  the  Effect  of  DoD  Acquisition  Policy  and  Process  on  Cost 
Growth  of  Major  Defense  Acquisition  Programs,”  IDA  Paper  P-5126.  Alexandria,  VA:  Institute  for  Defense 
Analyses,  September  2014.  Thispaperwas  sponsored  by  the  Director,  Performance  Assessments  and  Root  Cause 
Analyses  of  the  Office  of  the  Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics.) 
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Table  1  displays  the  average  PAUC  growth  for  MDAPs  that  passed  Milestone  (MS)  B  or 
(pre-2001)  MS  II  or  filed  a  first  Selected  Acquisition  Report  (SAR)  in  each  of  these  regimes.  The 
PAUC  growth  figures  all  are  measured  from  the  MS  II/B  baseline  and  normalized  to  the  MS  II/B 
total  inventory  objective.  There  are  a  number  of  interesting  aspects  to  these  data;  for  example, 
the  high  PAUC  growth  during  the  AR  period  and  the  lower  PAUC  growth  for  2001-2007. 
Granting  that,  the  single  most  notable  feature  of  these  data  is  the  absence  of  any  trend  in  PAUC 
growth.  If  changes  in  acquisition  policy  and  process  have  had  a  sustained  influence  on  PAUC 
growth,  it  does  not  show  up  in  this  table. 


Table  1.  Average  PAUC  Growth  in  Successive  Acquisition  Regimes 


Acquisition  Regime 

Time  Period 

Average  PAUC 
Growth 

No.  of 

Observations 

DSARC 

1970-1982 

32% 

48 

Post  Carlucci  Initiatives  DSARC 

1983-1989 

19% 

40 

DAB 

1990-1993 

36% 

11 

Acquisition  Reform  (AR) 

1994-2000 

66% 

27 

DAB  post  AR 

2001-2007 

19% 

25 

Broadly,  there  are  two  ways  to  explain  the  absence  of  sustained  effects  of  acquisition  policy 
and  process  on  the  PAUC  growth  data.  First,  they  may  in  feet  not  have  a  strong  or  consistent 
effect  on  PAUC  growth.  Second,  acquisition  policy  and  process  may  have  substantial  effects  that 
are  masked  by  some  other  factor  or  factors. 

Funding  Climate  and  PAUC  Growth 

Thinking  along  the  lines  of  the  second  of  these  possibilities  led  to  consideration  of  whether 
changes  in  the  DoD  funding  climate  might  be  associated  with  PAUC  growth.  The  period  1970- 
2007  includes  two  sub-periods  during  which  the  DoD  budget  was  Relatively  Constrained: 
FY  1970-FY  1980  and  FY  1987-FY  2002.  It  also  includes  two  sub-periods  in  which  MDAP 
new  starts  found  funding  climate  Relatively  Accommodating:  FY  1981-FY  1986  and  FY  2003- 
FY  2007.  Table  2  displays  the  average  PAUC  growth  data  for  these  four  sub-periods. 


Table  2.  Average  PAUC  Growth  in  Different  Funding  Climates 


Relatively  Constrained 

Relatively  Accommodating 

Period  (FY) 

PAUC  Growth 

Period  (FY) 

PAUC  Growth 

1970-1980 

35%  (42) 

1981-1986 

12%  (35) 

1987-2002 

53%  (55) 

2003-2007 

7%  (19) 

Note:  Numbers  in  parentheses  are  the  numberofobservations  available. 


These  data  make  it  clear  that  the  average  PAUC  growth  in  Relatively  Constrained  funding 
climates  was  far  larger  than  it  was  in  periods  during  Relatively  Accommodating  funding 
climates — by  a  factor  of  three  in  the  first  comparison  and  by  a  factor  of  more  than  seven  in  the 
second. 
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Acquisition  Regime  and  Funding  Climate 

Table  3  expands  Table  2  by  replacing  the  funding  climate  sub-periods  with  the  acquisition 
policy  and  process  regimes.  This  table  provides  results  for  two  sets  of  natural  experiments.  First, 
the  PAUC  growth  columns  give  the  effect  of  changes  in  the  acquisition  regime  for  a  given 
fonding  climate.  Second,  the  rows  show  the  effect  of  funding  climate  for  a  given  acquisition 
regime.  For  example,  the  first  eleven  years  of  the  DSARC  (FY  1970-FY  1980)  were  in  a 
Relatively  Constrained  funding  climate,  while  the  next  two  (FY  1981-FY  1982)  were  in  a  period 
in  which  the  DoD  budget  was  Relatively  Accommodating. 


Table  3.  Average  PAUC  Growth  by  Acquisition  Regime  and  Topline  Condition 


Acquisition  Regime 

Relatively  Constrained 

Relatively  Accommodating 

Period  (FY) 

PAUC  Growth 

Period  (FY) 

PAUC  Growth 

DSARC 

1970-1980 

35%  (42) 

1981-1982 

11%  (6) 

PostCarlucci  DSARC 

1987-1989 

34%  (11) 

1983-1986 

13%  (29) 

DAB 

1990-1993 

36%  (11) 

None 

N/A 

Acquisition  Reform  (AR) 

1994-2000 

61%  (27) 

None 

N/A 

DAB  post  AR 

2001-2002 

57%  (6) 

2003-2007 

7%  (19) 

Note:  Numbers  in  parentheses  are  the  numberofobservations  available. 


Statistical  analysis  of  the  data  behind  the  averages  in  this  table  leads  to  two  conclusions. 
First,  there  is  no  statistically  significant  improvement  or  worsening  of  PAUC  growth  correlated 
with  the  different  acquisition  policy  and  process  regimes.  This  result  is  not  surprising  for  the 
Relatively  Accommodating  climate  (column  on  the  right).  In  contrast,  in  the  Relatively 
Constrained  periods  (column  on  the  left),  average  PAUC  growth  for  AR  and  DAB  post-AR  is 
noticeably  higher  than  the  averages  for  previous  periods,  but  the  differences  proved  not  to  be 
statistically  significant  because  of  the  large  variance  among  programs  in  each  period. 

Second,  average  PAUC  growth  was  substantially  higher  in  a  Relatively  Constrained 
funding  climate  than  in  the  Relatively  Accommodating  climate.  We  have  only  three  natural 
experiments  of  changes  in  funding  climate  for  a  given  acquisition  regime,  since  two  of  the  five 
acquisition  regimes  (DAB  and  AR)  fall  entirely  within  one  funding  climate.  Each  of  these  three 
natural  experiments  on  the  effect  of  funding  climate  had  the  same  outcome — MDAPs  that  passed 
MS  II/B  in  a  Relatively  Constrained  funding  climate  on  average  had  a  much  higher  PAUC 
growth  rate  than  those  that  passed  MS  II/B  in  a  Relatively  Accommodating  funding  climate  for  a 
given  acquisition  regime.  These  differences  are  statistically  significant  at  the  1  percent 
confidence  leveL  The  outcomes  of  the  first  two  experiments  are  virtually  identical — an  average 
PAUC  growth  of  35  and  34  percent,  respectively,  in  the  two  periods  when  funding  was 
Relatively  Constrained  and  average  PAUC  growth  of  11  percent  and  13  percent,  respectively,  in 
the  two  periods  when  the  funding  climate  was  Relatively  Accommodating.  The  effect  is  even 
more  pronounced  in  the  third  experiment  (DAB  post-AR) — 57  percent  in  FY  2001-FY  2002 
versus  just  7  percent  for  FY  2003-FY  2007. 
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Does  the  Resource  Allocation  Process  Play  a  Major  Role  in  PAUC  Growth? 

These  conclusions  tend  to  challenge  a  fundamental  assumption  implicit  in  most  discussions 
of  acquisition  refonn:  that  the  main  causes  of  PAUC  growth  are  to  be  found  in  the  acquisition 
realm — the  performance  of  the  contractors,  the  effectiveness  of  the  PM,  the  adequacy  of  the 
developmental  test  plan,  and  the  completeness  of  the  systems  engineering  plan,  among  others. 
This  assumption  is  hard  to  maintain  when  the  many  changes  in  acquisition  policy  and  process 
made  in  the  past  four  decades  have  not  had  statistically  significant  effects  on  PAUC  growth,  but 
there  is  a  significant  association  between  PAUC  growth  and  funding  climate  at  the  point  when 
the  MS  II/B  baseline  was  set. 

The  association  between  PAUC  growth  and  funding  climate  suggests  that  the  resource 
allocation  process,  particularly  at  the  Service  level,  plays  an  important  role  in  cost  growth.  This 
does  not  mean  “budget  instability.”  Budget  instability  is  a  tenn  of  art  for  changes  in  MDAP 
funding  through  the  annual  resourcing  cycle  and  “taxes.”  Budget  instability  is  a  chronic 
condition,  present  to  some  degree  in  all  periods.  What  this  paper  observed  is  a  recurring 
pattern — that  MDAPs  that  passed  MS  II/B  during  periods  of  Relatively  Constrained  funding,  on 
average,  had  much  higher  PAUC  growth  than  those  that  passed  MS  II/B  when  funding  was 
Relatively  Accommodating. 

The  conjecture  that  the  resource  allocation  process  plays  an  important  role  in  cost  growth 
gets  some  support  from  an  unexpected  direction — MDAPs  with  negative  cost  growth,  of  which 
there  are  twenty-nine  in  our  sample.  Negative  PAUC  growth  is  recorded  if  the  actual  cost  of  a 
program  proves  to  be  less  than  the  cost  in  the  MS  II/B  baseline.  Assuming  the  program  was 
funded  to  its  MS  II/B  baseline,  this  implies  that  over  time  funds  can  be  taken  from  the  program 
in  question  and  reallocated  to  other  applications,  including  other  acquisition  programs.  The 
program,  then,  effectively  can  be  used  as  a  “bank” — a  way  to  hold  reserves  in  relative  safety 
until  they  are  needed.  A  bank  of  this  sort  is  more  likely  to  be  needed  in  a  Relatively 
Accommodating  funding  climate,  as  it  can  then  serve  as  a  way  to  delay  final  decisions  on 
allocation  of  the  higher  level  of  funding  that  has  become  available.  We  would  therefore  expect  to 
find  a  higher  proportion  of  MDAPs  with  negative  PAUC  growth  in  the  Relatively 
Accommodating  climates,  and  this  is  what  we  observe.  About  30  percent  of  our  MDAPS  that 
passed  MS  II/B  in  Relatively  Accommodating  funding  climates  show  negative  PAUC  growth, 
compared  to  about  10  percent  across  the  periods  of  Relatively  Constrained  climate. 

MDAPs  with  “high  cost  growth,”  which  we  define  as  quantity  nonnalized  PAUC  growth  of 
at  least  50  percent,  also  suggest  an  influence  from  the  resource  allocation  process.  DoD  resource 
managers,  particularly  at  the  Service  level,  have  only  a  few  tools  for  responding  to  a  Relatively 
Constrained  funding  climate.  One  of  these  is  to  impose  top-down  limits  on  the  funding  for 
particular  MDAPs  as  they  approach  MS  II/B.  Plausibly,  the  result  will  be  particularly  optimistic 
programmatic  and  costing  assumptions,  which  lead  to  an  expectation  that  MDAPs  started  in 
periods  of  Relatively  Constrained  funding  climate  will  have  a  larger  proportion  with  high  PAUC 
growth.  This  is  again  what  is  observed.  During  periods  of  Relatively  Constrained  funding 
climate,  about  40  percent  of  MDAPs  had  very  high  PAUC  growth.  In  contrast,  during  periods  of 
Relatively  Accommodating  funding  climate,  only  about  7  percent  of  MDAPs  experienced  high 
PAUC  growth. 

Taking  both  funding  climates  together,  85  percent  of  MDAPs  with  PAUC  growth  of  at  least 
50  percent  passed  MS  II/B  during  a  Relatively  Constrained  funding  climate.  These  MDAPs  had 
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an  average  PAUC  growth  of  93  percent  and  accounted  for  just  over  three-quarters  of  total  PAUC 
growth.  Excluding  high  cost  growth  MDAPs  and  MDAPs  with  negative  PAUC  growth,  average 
PAUC  growth  across  the  two  funding  climates  was  just  18  percent.  High  PAUC  growth  is  then 
predominantly  a  feature  of  programs  with  PAUC  growth  of  at  least  50  percent,  and  these 
programs  mainly  passed  MS  II/B  in  periods  of  Relatively  Constrained  lunding  climates.  These 
points  are  important  because  they  suggest  that  reforms  directed  to  the  average  or  typical  MDAP 
may  miss  the  real  source  of  the  problem. 

Implications  for  Discussions  of  Acquisition  Reform 

This  paper  points  to  three  implications  for  a  discussion  of  acquisition  reform  First,  the 
relevant  context  for  understanding  PAUC  growth  is  the  interface  between  the  acquisition  process 
and  the  resource  allocation  process.  The  crucial  evidence  behind  this  point  is  the  strong 
association  between  lunding  climate  and  PAUC  growth.  Resource  managers  must  think  in  terms 
of  a  portfolio  of  programs  at  various  stages  of  the  acquisition  life  cycle,  from  efforts  in  the 
technology  base  through  programs  nearing  the  end  of  production.  When  a  program  is  completed, 
it  opens  a  resource  “hole”  that  programs  emerging  from  Engineering  and  Manufacturing 
Development  can  occupy.  In  turn,  programs  earlier  in  the  acquisition  cycle  can  move  forward  as 
well.  When  Holding  for  acquisition  turns  down,  these  holes  get  smaller,  or  close  entirely,  or 
require  cuts  in  lunding  for  ongoing  programs.  The  alternatives  available  in  this  circumstance  are 
all  undesirable — cancellations  of  programs,  delays  in  new  starts,  stretches,  and  unrealistic 
pricing.  The  evidence  summarized  here  suggests  that  it  is  in  this  context  that  high  PAUC  growth 
arises. 

Second,  it  seems  unlikely  that  further  broad  changes  in  the  acquisition  process  would  have 
a  major  effect  on  PAUC  growth.  The  research  found  no  evidence  that  the  efforts  to  strengthen 
the  acquisition  process  through  the  years  have  resulted  in  lower  or  higher  PAUC  growth.  This 
does  not  mean  that  the  DAB  process  does  not  provide  a  useful  discipline  on  acquisition 
programs;  moreover,  further  changes  in  acquisition  policy  or  process  might  be  warranted  for 
reasons  of  good  government.  The  evidence  does,  at  a  minimum  suggest  that  the  effects  of 
changes  in  the  acquisition  process  since  the  early  1970s  have  not  had  a  dominant  effect  on 
PAUC  growth. 

Third,  it  is  difficult  to  see  that  the  cultures  of  the  DoD  acquisition  organizations  are  a 
crucial  obstacle  to  improved  performance  on  cost  growth.  The  key  point  to  note  is  that  high 
PAUC  growth  is  not  persistent,  but  rather  episodic,  and  correlated  with  environmental  factors 
outside  of  the  control  of  the  acquisition  process.  There  is  remarkably  little  PAUC  growth  in 
periods  when  the  funding  climate  is  Relatively  Accommodating.  It  seems  fair  to  ask  if  it  makes 
sense  to  assert  that  an  entrenched  culture  sometimes  results  in  high  cost  growth  and  other  times 
in  low  cost  growth.  Just  how  is  it  that  the  A  team  takes  the  field  so  quickly  and  quietly  when  the 
budgetary  sun  comes  out?  And  why  even  in  bad  budgetary  weather  do  more  than  half  of  MDAPs 
exhibit  comparatively  modest  PAUC  growth? 
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